Learning in Feedforward Neural Networks Accelerated by Transfer Entropy

Maximum Information Transfer in Feedforward Neural Networks

The principle of maximum information preservation has been successfully used to derive learning algorithms for self-organizing neural networks. In this paper, we state and apply the corresponding principle for supervised networks: the principle of minimum information loss. We do not propose a new learning algorithm, but rather a pruning algorithm which works to achieve minimum information loss ...

Learning Stochastic Feedforward Neural Networks

Multilayer perceptrons (MLPs) or neural networks are popular models used for nonlinear regression and classification tasks. As regressors, MLPs model the conditional distribution of the predictor variables Y given the input variables X . However, this predictive distribution is assumed to be unimodal (e.g. Gaussian). For tasks involving structured prediction, the conditional distribution should...

Avoiding Local Minima in Feedforward Neural Networks by Simultaneous Learning

Feedforward neural networks are particularly useful in learning a training dataset without prior knowledge. However, weight adjusting with a gradient descent may result in the local minimum problem. Repeated training with random starting weights is among the popular methods to avoid this problem, but it requires extensive computational time. This paper proposes a simultaneous training method wi...

Distributed learning algorithm for feedforward neural networks

With the appearance of huge data sets new challenges have risen regarding the scalability and efficiency of Machine Learning algorithms, and both distributed computing and randomized algorithms have become effective ways to handle them. Taking advantage of these two approaches, a distributed learning algorithm for two-layer neural networks is proposed. Results demonstrate a similar accuracy whe...

Selective Learning for Multilayer Feedforward Neural Networks